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(54) Predictive and pulsed Illumination of a surface In a mtero-tarturo navigation technique 



(57) An optical navigation system save* power by 
pulsing Hs surface Illumination light source (3) to provide 
light only when needed. The level of light may be con- 
trolled by a aorvo-mec nanism that monitors degree of 
correlation data and average Illumination, end that 
changes the controlled level at times when It Is both 
required and safe to do so. As for cresting different lev- 
els of Illumination, that may be accomplished by: [1] 
pulsing the light on (23) and than opening the electronic 
Bhutter tor some period of time (24) during the pulse 
(gating the light); (2] opening the shutter and then vary- 



ing the pulse width of me light (21 , 22) (gating the LED); 
[3] opening the shutter end then pulsing the LED at dif- 
ferent Intensities (19, 20); and [4j a combination of [2] 
and [3]. in addition, the velocity or the navigation circuit 
over the surface (or vise versa) can be determined, and 
In cases where It Is safe to assume that limited acceler- 
ation la possible, at low velocities It Is desirable to 
reduce the rate of data acquisition, and thereby pulse 
the tight sources less often and conserve power. 
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Description 

Incorporation* By Reference 

s [0001] Trie subject matter of the present application Is related to that disclosed In US. Patent 5. 644, 139. entitled 
NAVIGATION TECHNIQU E FOR DETECTING MOVEMENT OF NAVIGATION SENSORS RELATIVE TO AN OBJECT 
(Issued 1 July "97), TO that disclosed In U.S. Patents, 729, 008, entitled METHOD AND DEVICE FOR TRACKING REL- 
ATIVE MOVEMENT BY CORRELATING SIGNALS FROM AN ARRAY OF PHOTO ELEMENTS (Issued 1 7 March m 
and hJbo to that disclosed In U.S. Patent Applications 00/052, 046 entitled "SEEING EYE' MOUSE FOR A COMPUTER 

w SYSTEM (filed 3D March W) end (S/N unknown) entitled EXPOSURE SERVO FOR OPTICAL NAVIGATION OVER 
MICRO-TEXTURED SURFACES, filed on the same day as the Instant application by Thomas C Ollvar and Brian L. 
Hastings, and assigned to Hewlett-Packard Co. In the Interest of brevity, U.S. Patents 5, 644, 139 & 5, 729, 008 and U.S. 
Applications 09/062, 048 and EXPOSURE SERVO FOR OPTICAL NAVIGATION OVER MICRO-TEXTURED SUR- 
FACES are each hereby Incorporated herein by reference. 

is 

Background Of The Invention 

[0002] A recently developed technique of tracking motion and accumulating position by correlating shifted 
Instances of a previous Image with new Images of surface micro-texture, such as the fibers In a sheet of paper, offers 

so ease of use and outstanding accuracy. Device* of these sorts are described in the incorporated patent documents. A 
generic term for such a technique Is "optical navigation', and It can reasonably be expected that It will come Into aver 
wider use as it becomes more widely known and appreciated Power consumption Is often an Issue In the types of appli- 
cations that optical navigation la suited tor. For example, a hand held scanner Is much more attractive as a battery pow- 
ered device than as one powered from the AO mains. A battery powered computer operated with a mouse that uses 

25 optical navigation Is a further example. 

[0003] The optical navigation technique depends upon Illumination. The photosensitive elements that ere used to 
capture the Image have an 'electronic shutter/ In that they can be continuously exposed to the Illuminated Image with- 
out benefit of a mechanical shutter. Ideally, they accumulate charge or otherwise transduce from light to an electrical 
value only In response to a signal to do so. Unfortunately, some photosensitive devices experience bras phenomena 

ao that Interfere with making measurements when the light Is pulsed on after being off. These devices can still be said to 
have an electronic shutter, but the nature of the devices dictates that the source of light be left on. There are however, 
types of photosensitive devices that can be disposed In arrays suitable for optical navigation, and that both have elec- 
tronic shutters end are less sensitive to the bias phenomena (e.g. r photo transistors, such as used In digital cameras), 
allowing the power savings of pulsed light operation. Our Interest herein is directed to an optical navigation system hav- 

3S Ing such an array of sensors. 

[0004] Consequently, early optical navigation systems simply supplied the desired Wumlnatlon continuously, even 
though It might be needed only for ffve or ten percent of the time. In addition, they continuously sample at a maximum 
rate. Ordinarily, simplicity fn a complex system is a welcome property, and In this case ft Is hot that continuous Illumina- 
tion and maximal sampling do not work; they do. and quite well, too. But unfortunately Illumination of the surface Is per- 

40 formed with IR (Infra Red) LEO'S, and Is a significant consumer of battery power, whloh la limited. It would be desirable 
If the optical navigation technique could be modified to cooperate with Illumination on demand without Interfering with 
Ha ability to reliably keep track of where ft Is. A reduction in sampling rets, where possible, would also be desirable, as 
It Is the taWng of the samples that Is the major consumer of power. 

<* SUMMARY OF THE INVENTION 

[0005] An optical navigation system eaves power by ^uldng Its surface Illumination light source to provide light only 
when needed The level of light may be controlled by a servo-mechanism that monitors degree of correlation data and 
average Illumination, and that changes the controlled level at rjnes when It Is both required and safe to do so. As for 

eo creating different levels of Illumination, that may be accomplished by. (1) pulsing the fight on and then opening the elec- 
tronic shutter for some period of time during the pulse (gating the light); (2) opening the shutter and then varying the 
pulse width of the light (gating the LED); (3) opening the shutter and then pulsing the LED at different Intensities; end 
(4) a combination of(2) and (3), In addition, the velocity or the navigation circuit over the surface (or vise versa) can be 
determined, and In cases where It Is safe to assume that limited acceleration la possible (e.g.. a hand herd scanner), at 

as low velocities It Is desirable to reduce the rate or data acquisition, and thereby pulse the light source less often and con- 
serve power. Trie reduction can be a calculated prediction of when It Is actually necessary to sample again, based on 
a worst case acceleration, or ft can simply be a conservative estimate that Involves suppressing normally scheduled 
pulses and honoring others. 
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Brief Description Of The Drawings 
[0006] 

5 Figure 1 la a simplified Woe* diagram Illustrating certain aspects or a system Incorporating optical navigation In 
which there Is a beneficial reduction In power consumption produced by controlling exposure with a pulsed a light 
source; 

Figure 2 la a elmplffled waveform diagram Illustrating a method of varying exposure for the optical navigation circuit 
of Figure 1 that relies upon adjusting LED drive pulse height; 
w Figure 3 la a elmplffled wsvetorm diagram Illustrating a method of varying exposure tor the optical navigation circuit 
of Figure 1 that relies upon adjusting LED drive pulse width; 

Figure 4 is a simplified wavetorm diagram illustrating a method of varying exposure forthe optical navigation circuit 
of Figure 1 that relies upon adjusting effective LED drive pulse width by gating the resulting light with an electronic 
shutter signal; 

is Figure 6 Is a sbnpllfled waveform diagram Illustrating a method of reducing power consumption for the optical nav- 
igation circuit of Figure 1 by controlling the rate of sampling according to the relative velocity; and 
Figure 6 Is a efrnpOfled waveform diagram Illustrating a method of reducing power consumption for the optical nav- 
igation circuit of Figure 1 by suppressing regularly scheduled samples to control the rate of sampling according to 
the relative velocity. 



20 



40 



Description Of A Preferred Embodiment 



p007] Refer now to Figure 1 , wherein b shown a simplified block diagram 1 of a system Incorporating optical nav- 
igation. In particular, it Is battery powered by a battery operated power supply 1B, although this Is not an essential 
ss requirement even circuits operated from the AC mains can benefit from reduced power consumption (longer compo- 
nent life, less heating, no fan, etc.) 

[0008] An optical navigation circuit 2, which may be as described or similar to those described In me Incorporated 
material, Is proximate a surface 6 whose mloro-texture fe Illuminated by one or more LED*s 3. Light 4 Prom the LEO'S 3 
Is reflected from surface 5, and some of that reflected light 6 enters an aperture (not shown) of the optical navigation 
w drcutt 2. The optical navigation circuit 2 registers changas In Its physical location relative to the surface 6 by tracking 
the apparent motion of micro-texture produced patterns In an array of photo-eensltlve devices (not shown) proximate 
the epBrture. Correlation techniques between present and past frames of pixel Images are used to do this, as explained 
in considerable detail In the Incorporated documents. We term this process of tracking motion "navigation. 11 It will be 
appreciated that the optical navigation circuit 2 Is an IC (Integrated Circuit) of considerable size, and that It has an 
35 amount of self-contained processing power commensurate with the task. It Is not by Itself an entire system, however, 
and Its outputs (6-12) are ueed by an outer feyeror processing tor some production of some result (13). 
[0009] Accordingly, note that several outputs from the optical navigation cl rcult 2 are applied to a system controller 
7 that does produce a system output 13 (e.g., a scanned Image or a position). The outputs from the optical navigation 
circuit 2 Include an average lUumlneilon value 8. correlation bowl data 9, Incremental motion signals 1 1 for X and Y 
axes, and other data 12. In addition, there are other control Inputs 10 that originate with the system controller 7, such 
as a reset signal, and other signals tor various housekeeping functions. 

[Ooio] A word is In order about the correlation bowl data signals 9, and we pause briefly to digress. Recall that we 
said that navigation Involves tracking apparent movement of a pixel pattern. This Is realized by comparing a data frame 
to a reference frame. The comparison Is made by shifting the contents of one frame relative to the other by one pixel In 

45 some direction and asking If the pixel values In the various locations are the same (or, is the sum of all their respective 
differences zero, or so). The comparison 'In some direction' is performed for a pixel displacement In all directions: one 
up, one down, one left, one right, one up and one left, one up and one right one down and one left, and, one down and 
one right. TTiat Is a total of eight comparisons. But we muetnt forget that there might have been no movement, so we 
also need result for no shifting. That makes n Ine comparison values that correspond In principle to what wo have termed 

x> correlation values. The one that Is the lowest represents the relative motion we are tracking. To be more robust In this 
process, we could go also to the next nearest pixel positions, which would raise the number of correlation values to 
twenty-five. Trie correlation process Is explained In considerable detail In the Incorporated documents. The "bowl' la 
simply the three dimensional plot of the correlation amplitudes versus the shifted position they represent. The signifi- 
cance of the bowl, besides Indicating movement or the lack thereof, Is that Its shape gives rise to confidence that the 

ss navigation process Is working as It is Intended to. For example, If the bowl Is flat" we can presume that there Is not 
enough contrast amongst the pixels to navigate upon. That Is a serious situation that Is to be avoided during navigation 
, Lack of excursions In the bowl may be brought on by changes In the surface as navigation proceeds, and mloht be cor- 
rectable by changes In expceu re. 
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[0011] lb resume, then, one subsystem In the system controller 7 to of particular interest to us. It Is a predictive 
sample and exposure controller 14, whose outputs are an LED drive signal 15 and a shutter signal 18. The LED drive 
eigne) 1 S wfll be a pu fee of vertebra duration or amplitude, or perhaps both. It determines the amount of emitted light 4 
that Is available to become reflected light fi, a portion of which may subsequently enter the aperture of the optical nav- 

5 Igatlon cfjcutt 2. The amount or reflected light that actually reaches the array of photo detectors (not shown) and that 
Influences their output, Is termed the "exposure/ If there were no electronic shutter mechanism (which Is perhaps 
unlikely, but Is possible) then the light reaching the array would indeed be solely responsible for the level of exposure. 
More probably there Is an electronic shutter whose "open time" Is determined by the shutter signal 1 6, which Is also a 
pulse. That pulse Is applied to an LED power driver 1 7 that actually drfvea the one or more LEO'S 3. from which light 4 

to le emitted Topically then, the resulting exposure will be a function of the LED drive signal 15 and the shutter signal 1 6. 
[001 2] Now, It Is not our Intent to describe any particular manner of determining whet the exposure ought to be. 
There are conventional way9 of doing this, In addition to which there Is the technique described In the U.a Patent Appli- 
cation entitled EXPOSURE SERVO FOR OPTICAL NAVIGATION OVER MICRO-TEXTURED SURFACES, mentioned 
and Incorporated herein at the etart of this application. It Is sufficient here to point out that such exposure level determl- 

is nation probably miles upon average Illumination 8, and preferably also upon the correlation bowl date 9. We are, how- 
ever, Interested In how to produce a particular exposure once It Is known that a particular exposure level Is desired, or 
that an existing exposure Is to be increased or decreased. 

[0013] Refer now to Figure 2, wherein le shown a method of varying exposure with different amplitude LED drive 
pulses. In particular, pulse 1 9 has a high amplitude, and produces a correspondingly higher exposure tor Its associated 

6 data frame (sample) then does the pulse 20 having a lower amplitude. They occur at different times. Indicating that 
sometime during the Interim the predictive sample and exposure controller 1 4 decided to reduce the level of exposure. 
In this example arty electronic shutter Is assumed to be open during the durations of pulses 19 and 20. 

[0014] A related but different method Is depicted In Figure 3. Here the situation Is as described for Figure z, except 
that the varying quantity Is the width of the LED drive pulses 2 1 and 22, 

25 [0016] Still another exposure control method Is depleted in Figure 4. Here the LED 3 Is on for the entire duration of 
LED drive pulse 23, but the electronic shutter le open for a lesser duration Indicated by signal 24. In this example the 
shutter signal 24 Is contained within the duration of LED drive pulse 23, but they might overlap In a different way. In any 
event, It b their period of simultaneous existence that produces an effective LED pulse width 27 by the shutter's effect 
of restricting the photo-sensitive elements to the tlmo period Indicated by dashed lines 25 and 26. 

so [001 6J The above are ways of ■throttling 0 tha amount of exposure In a pulsed light environment, given that the pre- 
dictive sample and exposure controller 14 has decided how much to ■move the throttle,' Pulsing the light saves power 
by not consuming power to fire up the LEDfs) when they are not needed between samples. Note that It does not require 
any change to an existing scheme for deciding how often samples are to be taken. 

[001 7] There is yet another technique that can be used to effect such savings, and that is to control the rate at which 
as samples are taken. This may be practiced Independently, or In conjunction with one of the exposure control techniques 
set out above. This technique la shown In Figures 5 and 6. These techniques are based on the knowledge that a certain 
length of time Is required to take a sample frame and perform correlation, and that there Is on upper limit on acceleration 
that can be expected as well as on the permitted relative velocity between the navigation drcult and the surface. This 
knowledge can be used to reduce the sampling rote when the relative velocity le low. or zero. 
40 [001 a] Suppose, for example, It le zero. Suppose also thatthe reference frame In use has most or all of Its Bbllhy to 
be shifted Intact Then we can determine with assurance a minimum sampling rate that will guarantee, even If we dont 
sample during some Initial movement, that the next data frame will still be correlatabte with that reference frame. By 
reducing the sampling rate to that value we save power by avoiding unneeded uses of the Illuminating LED's 3. Now, If 
we keep track of relative velocity, we can adjust that minimum sample rate upward as needed to preserve that same 
46 advantage, aJthough a lesser savings In power will be possible. It may be necessary to consider how many fihtfte tho 
reference frame has experienced, and derate the reduction In sampling rale -tor that, as well. 

[0O1S] Now consider Figure 6. It deplete a first train of pulses 27, which represent campling (with whatever expo- 
sure Is set) at a relative high rate, perhaps even a maximal rate. This Is in contrast to the situation somewhat later In 
tine where sampling pulses 28 and 29 are Issued at a considerably lower rate. In this figure the actual sampling rate Is 

bo computed to be the minimum safe value, and Is essentially continuouely variable as a function of relative velocity. 

[0020] In Figure 6 a similar result Is obtained, but through a method that may be easier to Implement In some sys- 
tems. In thla method the baste repetition rate of the sampling pulses to fixed, but individual sampling pulses are gated 
to occur or not, as a stepped approximation of the technique of Figure 5. Thus, pulse train 30 represents a maximal 
sampling rate, while at a different point In time missing pulse 31 causes a lower sampling rate, while at yet another point 

sb in time missing pulses 32 and 33 cause a still lower sampling rate. 

[0021] One method of accomplishing the variable rate sampling described above Is to repeatedly compute e sam- 
pling rate based upon relative velocity and the associated parameter of system behavior, such as maximum allowed 
acceleration and maximum allowed velocity, all considered in light of the amount of apparent motion allowed between 
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a reference frame and a data fame (that Is, the amount of shifting that can be accommodated therebetween). A simpli- 
fication would be to partition the range of allowable velocities Into ranges, such as zero to one Inch per second, one to 
three Inches per second, three to nine Inches per second, and above. A look-up ta£Jo could then provide conservative 
safe values for the sampling rate at each of these velocities; thus saving the overhead of repeatedly re-calculating sam- 
pling rates. 

[0022] For example, consider an optical navfgatlon circuit having a forty-eeven by forty-seven of photo-sensors 
spaced on forty micron centers, irthe velocity were zero and we assumed a maximum acceleration of JSg, then In order 
to ensure less than one pbcel of motion between consecutive data frames we can allow at most e time Interval of about 
4 msec between frames. That Is: 

Jf = dspJecament = 1/2at 2 = pbcel spacing 

[0023] Solving fort 



75 



[0024] Where: 



20 



30 



a= 5 m/ eec z = V4 g and x = 40 w meters = one pixel 

[0025J "Hie maximum sample rate for the above system is about 1 00 usee Intervals between consecutive samples 
at a vetocBy of about .4 meters per second. Now, from 4 msec to 1 00 usee Is a torty-to-one change In sample rate, which 
represents quite a savings In power when the lower rata te possible. In this example one can, If desired, associate sam- 
ple rates with velocities by interpolating along the line having abscissas from zero velocity to .4 m/sec and correspond- 
ing ordinate* of 4 msec to 1 00 usee. As mentioned above, one could either repeatedly solve the equation for that line 
or (preferably) pre-load sample rates Into a look-up table Indexed by velocity. With a torty-to-one ratio, it would be appro- 
priate to partition the velocity Into perhaps as many as sixteen or thirty-two different entries In the look-up table. 

Palme 

1. A method of exposing to fight (4) a surface (5) having a micro-texture upon which an optical navigation circuit (2) 
tracks movement the method comprising the steps of: 



35 



40 



(a) determining a desired exposure level; and 

(b) Illuminating the surface wfth a pulse of light (10. 20) having an Intensity selected to produce the doslred 
exposure level. 

2. A method of exposing to light (4) a surface (5) having a micro-texture upon which en optical navigation circuit (2) 
tracks movement, the method comprising the steps of: 

(a) determining a desired exposure level; and 

(b) Illuminating the surface with a pulse of light ( 1 9, 20) having a duration selected to produce the desired expo- 
sure level. 



45 3> 



CO 



A method of exposing to Dght (4) a surface (5) having a micro-texture upon which an optical navigation circuit (2) 
tracks movement, the method comprising the steps of: 

(a) determining a desired exposure level; and 

(b) Illuminating the surface wfth a pulse of flght (23); and 

(c) operating In the optical navigation circuit an electronic shutter for a duration (24) thai Is selected to produce 
the desired exposure level, 

A method of exposing to light (4) a surface (6) having a micro-texture upon which an optical navigation circuit [2) 
tracks movement, the method comprising the steps of: 

(a) Illuminating th» surface with pulses of right (27) occurring at a selected rate; 

(b) determining e velocity from the tracked movement; 

(c) Increasing the selected rate In response to Increases In the determined velocity; and 
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(d) decrees trig the selected rate In response to decreases In the determined velocity, 

B. A method es In claim 4 wherein steps (0) end (d) ere computed Increases and decreases, respectively. 

5 6. A method as In clafrn 4 wherein the respect hve Increasing and decreasing of steps (c) and (d) comprise obtaining 
sample rate date from e table Indexed by the velocity determined In step (b). 

7. A method as In daJm 4 where fn step (a) nominally occurs at flxed periodic and minimal Intervals and step (d) com 
prteee euppresslng a pulse of light (31 , 32, 33) tn the nominally fixed periodic and minimal mtervale. 

to 
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FIG.2 (l£D PULSE AMPLITUDE) 



HIGH EXPOSURE 
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FIG.3 (LED PULSE WIDTH) 
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EFFECTIVE LED PULSE 
| WIDTH 27 
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FIG.4 (GATE LIGHT WITH SHUTTER) 
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FIG.5 (VARIBLE RATE SAMPLING) 



FIG.6 (SUPPRESSED CYCLE SAMPLING) 
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